MULTIGRADIENT  FIELD  ACTIVE  CONTOUR  FOR  MULTILAYER 
DETECTION  OF  ULTRASOUND  RECTAL  WALL  IMAGE 


Di  Xiao1,  Wan  Sing  Ng1,  Udantha  R.  Abeyratne2,  Charles  Bih-Shiou  Tsang3 

'School  of  Mechanical  &  Production  Eng.,  2School  of  Electrical  &  Electronic  Eng., 
Nanyang  Technological  University,  Singapore 
3Dept.  of  Colorectal  Surgery,  Singapore  General  Hospital,  Singapore 


Abstract- This  paper  presents  a  novel  multigradient  field  active 
contour  algorithm  with  an  extended  ability  for  multiple  object 
delineation,  which  overcomes  some  limitations  of  ordinary 
active  contour  models.  One  of  the  aims  is  to  apply  this  technique 
for  multilayer  boundary  detection  of  ultrasound  rectal  wall 
image,  which  is  important  in  colorectal  clinical  diagnosis  for 
rectal  tumor  staging.  The  core  part  in  this  algorithm  is  the 
proposal  of  multigradient  vector  field  concept,  which  is  used  as  a 
image  forces  for  alternate  constraints  on  deformation  of  the 
active  contour.  Its  application  on  the  ultrasound  rectal  wall 
image  is  also  given  to  illustrate  the  multiple  layer  detection 
ability. 
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I.  Introduction 

Colon  and  rectal  cancers  have  emerged  as  common 
diseases  in  the  developed  country  [1],  Like  other  cancers,  the 
early  stage  diagnosis  of  the  colorectal  cancer  is  more 
important  for  getting  good  treatment.  Currently,  one  of  the 
most  common  techniques  for  screening  test  and  early  staging 
of  the  rectal  tumors  is  endoscopic  ultrasonography  (EUS)  [2], 
which  can  obtain  a  cross-sectional  image  of  the  rectal  wall.  In 
order  to  extract  the  positional  information  of  the  rectal  tumor, 
surgeons  need  to  view  the  ultrasound  images  slice  by  slice, 
then  work  out  a  3D  mental  structure  of  rectum,  and  finally 
make  a  clinical  diagnosis.  So,  the  whole  procedure  is  a  time- 
consuming  and  tedious  work  for  surgeons.  On  the  other  hand, 
to  our  knowledge,  there  are  still  no  image  processing  methods 
that  have  been  described  in  the  literature  to  help  surgeons 
complete  this  task. 

The  goal  of  our  project  is  to  develop  an  automatic  analysis 
system  to  aid  surgeons  to  cope  with  large  quantities  of  rectal 
ultrasound  images,  involved  rectal  layer  detection,  rectal  wall 
tumor  detection,  3D  reconstruction  of  rectal  anatomic 
structure  and  corresponding  quantitative  analysis.  However, 
the  shape  and  layer  structure  detection  based  on  2D  rectal 
wall  images  is  an  initial  and  important  step  to  get  the 
necessary  information  for  3D  reconstruction.  In  this  paper, 
we  will  focus  on  the  algorithm  for  multilayer  detection  of 
rectal  wall  structure. 

The  layer’s  boundary  detection  of  rectal  wall  is  difficult 
because  of  the  low  spatial  resolution  of  ultrasound  images, 
the  thin  layer  structure  and  the  absence  of  the  layer  segment 
penetrated  by  the  tumor.  Therefore,  conventional  approaches, 
such  as  low-level  edge  detection  and  edge  linking  methods 
etc.,  are  hare  to  complete  our  task.  Other  high-level  methods 
do  not  meet  the  requirement  for  multilayer  detection  of  the 


rectal  structure  [3,  4],  In  the  recent  years,  elastic  deformable 
model  (snakes)  is  one  appreciated  algorithm  used  by  many 
researchers  for  boundary  detection  in  image  processing  [5,  6, 
7,  8],  The  name  “deformable  models”  is  originated  from  the 
elastic  theory  in  physics.  The  whole  object  matching  is  a 
searching  procedure  that  uses  the  optimal  approximation 
theory  to  obtain  the  energy  minimum  [9] .  In  this  project,  with 
snakes  method,  the  anatomical  knowledge  of  the  rectum  can 
be  integrated  into  layer  detecting  procedures.  Due  to  the 
characteristic  of  closed  active  contour,  snakes  method  can 
obtain  a  full  closed  boundary  despite  the  existence  of  the 
broken  layers.  Furthermore,  the  snakes  method  can  fulfil  the 
boundary  detection  and  object  recognition  at  same  time, 
which  cannot  be  done  by  low  level  method.  But,  some 
inherent  shortcomings  in  “snakes”  algorithm  limit  its 
application  in  this  project.  In  order  to  automatically  outline 
each  muscular  layer's  boundary  of  the  rectal  wall,  the 
approach  followed  in  this  paper  will  introduce  a  novel 
multigradient  field  active  contour  model  developed  by 
authors,  based  on  the  framework  of  elastic  deformable  model. 


II.  Multigradient  field  active  contour  model 

In  order  to  display  the  full  anatomical  structure,  the  muscular 
layer  detection  need  to  complete  the  a  few  tasks,  such  as 
border  detection  and  object  recognition  etc.  A  basic  idea  is  as 
follows.  Firstly,  set  up  an  initial  active  contour  in  the  balloon 
area  of  the  rectal  image.  Then,  expand  this  initial  model  to 
obtain  the  inside  boundary  of  the  innermost  layer. 
Subsequently,  use  this  boundary  as  the  new  contour  and 
continuously  expand  to  extract  the  next  layer’s  boundary. 
This  method  is  repeated  to  detect  each  muscular  layer  from 
inside  to  outside. 

A.  Multigradient  Field  Active  Contour  Model 

We  have  set  up  a  novel  active  contour  model  based  on  the 
framework  of  the  traditional  deformable  model,  as  follows: 

^  snake  [£int(v(s))  + 

^  image  ^  image  (v(s)) 

J  (I) 

+  wadp,Eexp(v(s))ds 

where,  v(s)  =  [x(s),  y(s)]  represents  the  deformable  model. 
Internal  energy  is  the  same  as  that  in  the  traditional  snakes, 

EjntCvCi))  =  j" ((ctr(s')|vJ, (^)|  +/?(s)|vm(,s)|2)/2  which  is  used 
to  compose  the  regularity  of  the  curve  ( a  and  /3  are 
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parameters  to  constrain  the  snakes’  tension  and  rigidity,  and 
vs(,s)  and  vsl(,v)  are  first-  and  second-derivatives  with 
respect  to  arc  length  s ).  Image  energy  Emw,,r  is  an  external 

energy  that  comes  from  the  image  features  so  that  it  takes  on 
its  smaller  values  at  the  features  of  interest.  Here,  a  novel 
multigradient  field  algorithm  is  proposed  to  compute  the 
multigradient  vectors  as  features.  Adaptive  expanding  energy 
Eex  is  an  external  energy.  The  corresponding  force  derived 

from  it  can  make  the  active  contour  model  expanded  with 
adaptive  adjustable  feature,  especially  in  the  homogeneous 
region  that  has  no  gradient  field.  W  is  weighting  parameter. 

Using  the  calculus  of  variations  and  solving  the  Euler 
equations  give  the  minimization  of  above  equation  [9].  In  the 
following,  only  two  important  formulae  are  given  for  further 
discussion  and  comparison.  The  Euler  equations  obtained 
from  the  Eq.  (1)  can  be  described  as  following  two 
independent  parts  according  to  x  and  y  coordinates: 
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We  can  use  finite  differences  to  derive  approximations  to  the 
differential  and  double  differential  of  the  above  function. 
Substituting  these  in  the  above  equations  will  give  a  pair  of 
systems  of  equations  for  finding  X  and  y  coordinates  that 


minimizes  the  energy  function  of  the  active  contour.  Each 
point  in  the  active  contour  can  be  converted  to  vector  notation 
with  v,  =  (Xj  y, ) .  The  final  numerical  solution  is, 


^ t  (C  +  /I)  (/x?_i  Wjmagef img_x (x?. _i , y t~\ )  (3a 

—  ^  adpfi  adpt_x  U  ,  | .  y  /  1 )) 

St~  (U  +  /1)  (ly t_i  —  w f jmg  y t-1  ’  Y t-i )  (3b 

—  ^arf/?t^adpt_y  (^  M  ’  Y  t-l )) 

Where^  fimg_xi 0  =  ^Eimage(X,  y)ldxt , 

and  fimg _y(0  =  dEimage(x,y)/dyi  are  image  forces; 
and  f adpt _  x  (0  =  dEadpt(x,  y)/dxj  and 

fadpt  v(0  =dEadpt(x,y)ldyj  are  adaptive  external  forces.  C 

is  a  pentadiagonal  matrix  including  some  parameter  infor¬ 
mation  such  as  a  and  / 3  ,  and  /  is  a  step  size,  t  denotes 


time  step,  while  x,  and  y,  are  vectors  including  the  X  and 
y  co-ordinates  of  the  active  contour  on  time  t . 


The  idea  through  the  approach  is  to  detect  the  rectal  layer 
one  by  one  from  inner  to  outer  by  deformable  model. 
Therefore,  when  one  layer  is  delineated,  the  algorithm  should 
be  able  to  deform  the  active  contour  again  and  search  the  next 
layer  automatically.  But  during  the  experiment,  it  is  found 
that  after  detecting  the  inner  boundary  the  active  contour 
model  would  not  expand  as  expected.  This  is  precisely 
happened  in  traditional  snakes  algorithms  -  once  an  active 
model  arrives  at  the  energy  minimum,  it  will  not  deform 
again. 

By  analyzing  traditional  snakes  formulae,  we  can  find  that 
image  energy  is  a  gradient  vector  after  performing  a  gradient 
operation  on  the  original  image.  However,  the  image  force, 
which  drives  the  deformation  of  the  active  contour,  is  a 
gradient  vector  obtained  from  the  gradient  operation  of  the 
image  energy  component.  It  means  that  image  force  is  a 
second-order  derivative  derived  from  image. 

In  order  to  explain  the  multigradient  field  algorithm, 
following  two  kinds  of  gradient  vector  fields  are  defined  to 
clarify  the  purpose  (A  ring-shaped  image  (a  black  ring  on  a 
white  background)  is  used  as  an  example).  (1)  First-order 
gradient  vector  field  (FOGVF)  -  Gradient  vector  field 
obtained  from  the  gradient  operation  on  the  original  image. 
The  corresponding  image  is  called  first-order  gradient  image 
(map);  (2)  Second-order  gradient  vector  field  (SOGVF)  - 
Gradient  vector  field  obtained  from  the  gradient  operation  on 
the  first-order  gradient  image.  The  corresponding  image  is 
called  second-order  gradient  image  (map). 

Fig.  1  shows  these  two  different  gradient  vector  fields 
operating  on  above  ring-shaped  image.  From  Fig.  1  (b),  it  is 
found  that  the  edge  of  the  ring  is  constrained  by  two  opposite 
forces  -  centripetal  force  and  centrifugal  force.  These  two 
forces  act  as  image  forces  that  have  the  active  contour 
attracted  by  edge  and  kept  in  the  status  of  energy  minimum. 
On  the  other  hand,  the  centripetal  force  is  just  the  force  that 
prevents  the  active  contour  from  expanding  and  searching 
outer  border  according  to  our  original  thought.  However, 
from  the  first-order  gradient  vector  field,  we  can  find  that  the 
centripetal  gradient  vector  in  the  inner  border  disappears 
because  of  the  gradient  feature  of  the  original  image.  This 
feature  is  exactly  what  we  want,  by  which  the  active  contour 
escapes  from  the  gravity  of  the  inner  border. 
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(a)  Gradient  vector  field  based  on  (b)  Gradient  vector  field  based  on 
original  image  -  first-order  gradient  gradient  image-second-order  gradient 
vector  field  vector  field 


B.  Multigradient  Vector  Field  Model 


Fig.  1.  First-order  gradient  vector  field  and  second-order  gradient  vector  field 
of  the  ring-shaped  image. 


Based  on  above  analysis,  a  novel  active  contour  algorithm 
incorporating  multigradient  vector  field  (multi-GVF) 
algorithm  is  developed.  This  model  named  multigradient  field 
active  contour  model  can  complete  the  multilayer  border 
detection  of  the  rectal  wall  structure.  Fig.  2  shows  how  to 
detect  the  inner  border  and  outer  border  of  a  ring  by  this 
algorithm.  A  detailed  procedure  about  the  algorithm  is 
described  as  follows: 

1 .  Compute  the  FOGVF  and  SOGVF  of  the  image. 

2.  Under  the  constraint  of  the  SOGVF,  deform  the  active 
contour  model  and  obtain  the  inner  border  (Fig.  2  (a)). 
The  corresponding  numerical  equations  are, 

—  (C  + /I)  (lx  —  W jmage^^ image  (Xf-1  >  y  r-1  )/3x?_j 
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y  t  ~  dy  /-I  Wjmage*^ image  (^r-1  ’  Y r- 1  V^Y /  ! 

—  ^  (/(//)^  adpt_y  r-1  ’  y  r-1 )) 

3.  Use  the  above  result  as  the  new  initial  contour,  deform 
the  active  contour  model  again  under  the  constraint  of 
the  FOGVF.  The  corresponding  numerical  equations 
become 

^r  —  (C  + /I)  (lx  ?_|  ^ bnage^ image  (Xf-py r-1  )  (5a) 

—  'V/d/j/f  adpi_x  r-1  >  y  r-1 )) 

y  r  —  (fy  r-1  -  WWEimage  (x  r-U  Y  r-1  )  (5b) 

—  'V/d/x/f  adpl_y  ^  X  /  I  ■  Y  /  I  )) 

As  the  gradient  vector  field  is  first-order  GVF,  the  energy 
minimum  position  that  deformable  model  reached  is  not  the 
real  edge  of  the  ring,  as  shown  in  Fig.  2  (c). 

4.  This  step  is  a  fine-tuning  or  refinement  of  the  final 
border  position.  The  SOGVF  is  used  again  like  step  2, 
and  the  contour  model  continues  to  deform  mainly  under 
the  second-order  gradient  vector,  the  internal  force  and 
the  adaptive  expanding  force.  Finally,  when  energy 
minimum  is  reached,  the  accurate  location  of  the  outer 
border  is  obtained  (Fig.  2  (d)). 


(a)  Extract  inner  boundary  under  SOGVF 


(b)  Escape  from  the 
attraction  of  Inner 
border  under  FOGVF 


(c)  The  intermediate 
result 


(d)  Extract  the  outer  boundary  under  SOGVF 
again. 


If  there  are  more  circular  layers,  steps  3  and  4  can  be 
repeated  to  complete  the  detection  of  each  layer. 

III.  Experimental  results 

The  pictures  given  in  Fig.  3  demonstrate  the  deformation 
process  and  the  results  of  multigradient  field  active  contour 
for  boundary  detection  for  different  layers  of  rectal  wall. 
According  to  the  multigradient  field  algorithm,  actually,  three 
different  gradient  vector  fields  and  their  corresponding 
gradient  maps  are  computed  and  used  as  image  forces  in 
different  detecting  phase.  They  are: 

•  First-order  gradient  vector  field  and  first-order  gradient 
map  of  original  image; 

•  Second-order  gradient  vector  field  and  second-order 
gradient  map  of  original  image; 

•  First-order  gradient  vector  field  and  first-order  gradient 
map  of  the  inverted  image  whose  grays  are  assigned  their 
mirror  transparency. 

Fig.  3  (a)  shows  the  detection  for  the  inner  boundary  of 
rectal  wall,  while  Fig.  3  (b)  shows  inner  boundary  of  the  first 
layer  of  rectal  wall.  From  the  result,  we  can  find  that 
multigradient  field  algorithm  can  give  a  good  detection  of  the 
edge  because  of  relatively  good  edge  features.  The  adaptive 

force  derived  from  WlldplEexv  actually  supplies  a  more 

flexible  and  stable  control  for  the  deformation  of  the  active 
contour  than  traditional  algorithm. 

Deformation  starting  from  the  location  shown  in  Fig.  3  (b) 
and  under  the  control  of  two  different  gradient  fields- 


(a)  Searching  under  SOGVF 


(c)  Outer  boundary  of  the  first  layer 


(b)  Inner  boundary  of  the  first 


(d)  Inner  3  layer’s  boundaries 


Fig.  3.  Experiment:  detect  the  probe  boundary  and  three  layers'  boundary  of 
the  rectal  wall  (Note:  Except  (a)  is  an  original  grey-level  image,  the  other 
images  are  diluted  as  background  of  each  picture  for  convenient  view  of  the 
delineated  boundary). 


Fig.  2.  Inner  layer  and  outer  layer  detection  by  two  different  GVFs 


FOGVF  and  SOGVF,  the  deformable  model  can  successfully 
extract  the  outer  boundary  shown  in  Fig.  3  (c).  In  the  whole 
procedure,  FOGVF  from  original  image,  which  can  help 
active  contour  escape  from  the  capture  range  of  the  inner 
boundary,  controls  the  first  stage  deformation  and  obtains  a 
temporary  result,  which  lies  between  the  inner  edge  and  the 
outer  edge  of  the  first  layer. 

In  the  second  stage,  active  contour  deforms  under  SOGVF 
and  finally  detects  the  outer  boundary.  Fig.  3  (d)  shows  the 
boundaries  of  the  inner  three  layers  obtained  by  multigradient 
field  algorithm.  Though  the  second  layer  is  very  thin,  it  is 
found  that  the  new  algorithm  can  accurately  distinguish  and 
extract  the  second  layer.  Of  course,  in  some  segment,  because 
of  rather  thin  layer  and  the  original  image’s  quality,  there  are 
some  overlapping  parts.  The  parameter  selection  for  the 
model  is  given  in  Table  1 . 


Table i 

Parameters  setting  for  mltigradient  field  active  contour 


Gradient  Vector  Field 

a 

P 

ML 

,v  image 

W adpt 

/ 

SOGVF 

0.024 

0 

20 

0.007 

1 

FOGVF 

0.030 

0 

2 

0.030 

1 

Three  surgeons  were  invited  to  evaluate  the  results  of  the 
algorithm.  Eight  image  slices  were  used  for  test.  By 
comparing  the  detected  boundaries  with  their  manual  tracing, 
the  surgeons  considered  that  the  detected  boundaries  were 
suitable  for  clinical  application.  A  further  quantitative 
assessment  like  accuracy  and  stability  of  the  algorithm  is  in 
progress. 

IV.  Conclusions  and  discussion 

The  investigation  of  rectal  wall  multilayer  delineation  on 
ultrasound  images  was  motivated  by  the  need  to  detect  and 
stage  the  tumor  of  the  rectum  for  clinical  application.  The 
first-order  gradient  and  second-order  gradient  vector  fields 
are  given  and  based  on  them,  a  novel  multigradient  vector 
field  algorithm  is  proposed  and  introduced  into  the  active 
contour  model.  This  algorithm  enables  the  active  contour  to 
escape  from  the  trammel  of  the  present  layer  and  continue  the 
next  layer  searching  and  finally  accomplish  the  multilayer 
detection.  Adaptive  force,  which  is  able  to  make  a 
compensation  between  internal  force  and  image  force, 
especially  in  homogeneous  regions  in  an  image,  also  plays  an 
important  role  in  the  practical  application. 

Several  problems  should  be  considered  in  the  future. 
Although  the  detection  results  generally  display  the  real 
positions  of  the  boundary  of  normal  layers,  the  detected 
boundaries  located  on  artifacts  and  tumor  area,  due  to  the 
effect  of  internal  force,  deviate  from  their  real  position. 


Another  problem  is  that  during  the  process  of  deformation 
the  active  contour  sometimes  is  easily  attracted  by  some 
"isolated  islands"  and  forms  an  irregular  border.  The  reason  is 
due  to  the  irregularity  of  the  boundary  and  some  small  broken 
segments  on  the  layer.  Pre-processing  the  image  may  be  able 
to  solve  these  defects  inherent  with  ultrasound  images.  In  this 
paper,  we  try  to  emphasize  the  general  features  of  the 
algorithm,  so  only  a  little  prior  knowledge  about  rectal 
anatomical  structure  was  incorporated  into  the  algorithm. 
More  prior  knowledge  and  constraints  may  be  incorporated 
into  the  basic  model  soon.  To  assess  the  performance  of  the 
automatic  boundary  detection  algorithm,  a  set  of  quantitative 
assessment  rules  should  be  set  up  and  the  further  experiments 
should  be  done. 
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